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Method and Apparatus for Distributing Routing Instructions over Multiple 
Interfaces of a Data Router 

5 Field of the Invention 

The present invention is in tlie field of data routing of multicast data packets 
over a data packet network, and pertains more particularly to assigning and 
distributing portions of a routing table to router line interfaces, the assigned portions 
10 pertinent to the data expected to arrive at the interface for forwarding. 

Cross-Reference to Related Documents 

The present invention is a continuation in part (CIP) to a U.S patent 
1 5 application S/N 09/854,234 entitled "Apparatus and Methods for Efficient 

Multicasting of Data Packets" filed on 05/10/2001, which is a CIP to a US Patent 
Application S/N 09/800,678, entitled "An Improved System for Fabric Packet 
Control", filed March 6, 2001, which documents are incoq)orated herein in their 
entirety by reference. 

20 

Background of the Invention 

With the advent of the well-known Internet network and similar data-packet- 
networks, much attention in the art has been devoted to improvement of packet 

25 routing technologies developed for routing data packets from source nodes to 

destination nodes, usually through multiple intermediary nodes or hops in a given 
neUvork topology, beUveen the source and destination locations. One of the most 
prevalent contnbutions to the art involves design and development of more efficient 
data routers. A state-of-the-art router known to the inventor makes use of a 

30 distributive architecture, having muUiple processors, enabling high scalability with 
respect to adding data routing capacity. 
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The router described above is known to the inventor as a Terabit Network 
Router (TNR), and has multiple line interfaces each having multiple ports for 
accepting data into the router and forwarding data out of the router. The TNR also 
has a fabric of intercormected processor nodes termed fabric cards for routing data 

5 through the router from line ingress to line egress. Multiple control processors in the 
TNR provide needed messaging fiinctions, configuration and protocol distribution, as 
well as special packet processing, among other functions. 

Making efficient use of bandwidth is an ever-present challenge to 
manufacturers of data routers. Eliminating unnecessary messaging between routers 

10 and reducing internal messaging and notification with respect to data packet 
processing is always a desirable goal in this respect. 

A method known to the inventor and used in a distributed processor router 
enables data management to be efficiently accomplished in a fabric network of the 
router without requiiing conventional flow control, which typically requires upstream 

15 propagation of flow control messages. The method known to the inventor provides 
for lower loss of data packets, and is less complex in operation than conventional 
flow-control methods. 

The known method involves establishing a virtual output queue (VOQ) and a 
queue manager at each incoming port path of each interconnected processor node or 

20 fabric card making up the fabric of the router. With respect to these nodes, each one 
has at least two, but typically more, external ports, and the individual ports of each 
node are coupled internally by a switcliing mechanism, termed a crossbar, to other 
ports of the node. The VOQ and queue manager provide management for data 
arriving at each port. In this data-routing system, all data is passed from an ingress 

25 port of the fabric card to an egress port of the fabric card as long as a queue in the path 
estabhshed between the ports is less than full. Ingress packets are discarded in a port 
path having a fiill queue until the queue level is again less than full. The system frees 
up processor resources from upsfream flow-confrol responsibility thereby reserving 
processor resources for other, more important data routing and management fiinctions. 
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Another area where more bandwidth management efficiency is desired is in 
the area of Internet Protocol (IP) multicasting. P Multicasting protocols, in general, 
work to reduce network traffic congestion by enabling delivery of a single data 
stream to multiple recipients, usually subscribers of the original stream. Applications 
diat take advantage of IP multicast technologies include video conferencing, IP 
telephony, educational interaction, file sharing, and the like. 

IP multicast technology is based on a group concept. For example, a group of 
network-connected nodes subscribes to a particular data stream. These nodes can be 
located anywhere on the connected network, and have no particular geograpliic 
limitations. Nodes that wish to receive data destined to a particular group of end 
nodes use a protocol termed Intemet Group Management Protocol (IG^IP). Such host 
nodes must be a group member of die Protocol in order to receive the stream. 

In current art, multicast data packets are replicated at routers, often usin» a 
well-known protocol termed in the art Protocol Independent Multicast (PIM). PIM 
can leverage prevalent network routing protocols to more efficiently route multicast 
data packets through a network. One of the more challenging tasks in IP multicastmg 
is the function of replicating the data packets for the multiple end destinations. 

There are, m broad terms, two categories of PIM. In one category, tenned 
Sparse mode (PIM SM), recipients are required to subscribe to the source; so in PIM 
SM the multicast streams are limited to the subscribers. There is also a Dense Mode 
(PIM DM), whereui packets from a source are sent to all reasonable destinations and 
those who do not want or need the packets simply drop the multicast packets. Clearly 
the DM mode results in a greater proliferation of data packets than the sparse mode, 
therefore the name. 

One liability inherent to prior and current art multicast methods as practiced on 
a data-packet-network is that routers that are enhanced for IP multicasting are 
typically not scalable in amount of multicast traffic they can handle, including the 
number of individual copies made of each multicast packet. Such routers are also 
lunited m the number of router ports available for forwarding multicast data. 
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An enliancement in IP multicast capability known to the inventor and utilized 
in distributed processor routers such as the TNR described above involves provision 
of a multicast engine for replicating packets incoming to the router and identified as 
multicast packets. The multicast enguie is distributed to assigned multicasting ports 
5 of fabric nodes or cards making up the internal routing fabric. The multicast engine at 
each assigned node uses a table that provides sets of instructions unique to each 
assigned port for completing its portion of a multicast assignment widi regard to 
numbers of copies made and internal addressing requirements, hi this way, multicast 
forwarding is distributed in a fan-out fashion through a network such that no 
1 0 concentrated use of network resources occurs, possibly creating overloads or 
bottlenecks. 

As described above, IP multicast addresses specify an arbitrary group (G) of IP 

hosts that have joined the group and wish to receive traffic sent to this group. In more 
granular implementation of PM, such hosts may also requests packets for G that 

15 originated from a particular source (S). However, when these downstream hosts 

receive and identify the multicast data at their ingress (upstream ports) a unicast, and 
in some cases a multicast routing table must be consulted in order to properly forward 
the packets to their next-hop or final destinations accorduig to the G metrics. 
Therefore, valuable processor resources operating in the hosts are diverted from 

20 nomial unicast processing in order to accomphsh the multicast forwarding 

requirements. Scaiming an entire table for instructions for processing every muhicast 
data packet arriving at ingress of the router is particularly burdensome for a 
distributed processor router having a large number of ingress ports receiving data. It 
has occurred to the inventor that an IP host, whether of the fomi of a distributed 

25 processor router or even a single processor router, could be made more efficient in 
temis of resource conservation if extensive table lookups for forwarding instructions 
could be avoided. 

Therefore, what is clearly needed is a mechanism for distributing just a portion 
of tlie larger body of IP multicast forwarding data to upstream line interfaces of a 
30 router tliat are expected to receive the multicast data that the distributed portion 
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applies to. A mechanism such as this would eliminate normally extensive data lookup 
processes that tax processing resources of a router engaged in IP multicast data 
forwarding. 

5 Summan' of the Invention 

In a preferred embodiment of the present invention a software application in a 
multi-processor data router in which a forwarding information base for the router is 
maintained is provided, comprising a server module, and one or more client modules, 

10 each client module associated with one or more communication interfaces of the data 
router. The application is characterized in that the server module sends to each client 
module only that portion of the forwarding information base specific to the 
communication interfaces associated with the client module. 

hi a preferred embodiment there is one server module and multiple client 

1 5 modules. Also in a preferred embodiment the operating protocol followed is protocol 
independent multicast (PIM). Further, an assigned number of physical port interfaces 
on a line processor may share a single block of forwarding information. Still further, 
the portions of multicast forwarding information may be periodically updated at tlieir 
client modules by the server module. Further yet, the router may be connected to and 

20 operates on the Internet network. 

In a preferred embodiment more than one portion of multicast forwarding 
information blocks distributed to a single client module are stored in a smgle 
forwarding table locally at the Ime processor, and individual ones of the physical 
interfaces access their assigned forwarding instmctions from the local table according 

25 to need. 

Still in a preferred embodiment there may fiirther a mechanism for asserting a 
drop state for unexpected multicast data packets arriving at any physical interface of 
an enabled line processor. There may also be a mechanism for creating forwarding 
state for unexpected multicast packets arriving at any of the physical interfaces of an 
30 enabled line processor. 
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IN another preferred embodiment, operating in protocol-independent 
multicasting sparse mode (PM-SM), a processor of the data router may send a 
request to a neighboring router to join a group, retrieving group and source 
information, and the sen'er module then prepares and sends to an appropriate client 
module that portion of the forwarding information base associated with packets 
expected to be received at the associated communication interfaces as a result of 
joining tlie group. 

In still another preferred embodiment the software application, operating in 
protocol-independent multicasting dense mode (PM-DM), each time an unexpected 
multicast packet is received at a communication interface associated witli a client 
module, the client module communicates infonnation about the packet to the server 
module, which identifies a portion of the forwarding information base pertinent to tlie 
packet, and then for^vards that portion of the forwarding information base to the client 
module. 

IN anotlier aspect of the invention a metliod for proccssmg multicast data 
packets in a multiple-processor data router is provided, comprising steps of (a) 
sending a request to an upstream router to join a multicast group, the request including 
an ingress interface for receiving the requested multicast data packets; (b) isolating 
and distributing a portion of a multicast forwarding infonnation base to a client 

) software module associated with the ingress interface expecting to receive tlie 

multicast data packets; (c) receiving the requested multicast data packets at the mgress 
interface; and (d) using the forwarded portion of the infonnation base to process the 
received multicast data packets. 

In one embodunent of the method, in step (a), the metrics include one or more 

5 of network layer address information, circuit ID metrics, and physical port metrics. In 
another embodiment, m step (b), the multicast forwardmg information follows 
protocol independent multicast (PIM), and distribution may be from a PIM server to 
one or more PM clients. 

In yet another aspect of the invention a method for processing multicast data 

,0 packets m a multiple-processor data router is provided, comprising steps of (a) 
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receiving a first multicast data packet of a data flow for forwarding at a line interface 
of the router; (b) initiating and sending a request to a control processor of the router, 
the request containing source and group information for the received data packet and 
requesting forwarding information; (c) receiving the request at the control processor 
5 and consulting with a main routing table to build a forwarding infomiation applicable 
to the data packet; (d) distributing the appropriate forwarding information to the 
requesting line interface; and (e) applying, at the line interface, the distributed 
infonnation to forward the multicast data packet and subsequent data packets of the 
same flow. 

10 In some cases, in step (b), the request is sent by a PIM client application to a 

PIM server application. Also in some cases, in step (d), the distributed forwarding 
information is stored in a local table at the line processor. The local the local table 
maybe updated periodically with new forwarding information. 

In yet another aspect of the invention a multicast-enabled data router is 

15 provided, comprising a plurality of processors, individual ones operating as clients 
and associated with specific ones of multiple coniniunication interfaces, and one 
functioning as a control processor having access to a for\^'arding information base, and 
a software application comprising a server module executing on the control processor, 
and one or more client modules executing on individual ones of tlie processors 

20 associated with specific communication interfaces. The router is characterized in tiiat 
the server module sends to each client module only that portion of the forwarding 
infomiation base specific to the communication interfaces associated with the client 
modules. 

hi a preferred embodiment of the router, the operating protocol followed is 
25 protocol independent multicast (PIM). In this and other embodiments the portions of 
multicast forwardmg information may be periodically updated at their cHent modules 
by the server module. The network can be the well-known Internet network. There 
may fiirther be a mechanism for asserting a drop state for unexpected multicast data 
packets arriving at any interface, and a mechanism for creating a forwarding state for 
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unexpected multicast packets arriving at any physical interfaces associated with a 
client module. 

In some cases the data router operates in protocol-independent multicasting 
sparse mode (PIM-SM), wherein a processor of the data router sends a request to a 
5 neighboring router to join a group, retrieving group and source information, and the 
server module then prepares and sends to an appropriate client module that portion of 
the forwarding information base associated with packets expected to be received at the 
associated communication interfaces as a result of joinmg the group. 

In another embodiment, operating in protocol-independent multicasting dense 
10 mode (PM-DM), each time an unexpected multicast packet is received at a 
conmunication interface associated with a client module, tlie client module 
communicates information about the packet to the server module, which identifies a 
portion of the forwarding information base pertinent to the packet, and then forwards 
that portion of the forwarding infomiation base to the client module. 
15 In yet another aspect a method for processing multicast packets is provided, 

comprising the steps of (a) sending, by a server module to a client module, the server 
module executing on a control processor and the client module executing on a 
processor associated with an individual communication interface, a portion of a 
forwarding infomiation base pertinent the communication interface; and (b) using, by 
20 the client module, the portion of the forwarding information base sent by the server 
module to process multicast packets received at the communication mterface without 
requesting mfomiation from the original forwarding information base. 

hi some cases of this there is one server module and multiple client modules. 
Further, the operatmg protocol followed may be protocol mdependent multicast 
25 (PM). Also further, the portions of multicast forwarding information are periodically 
updated at their client modules by the server module. Also, the method may be 
performed m a router connected to and operating on the hitemet network. Still 
fijrther, there may be included a mechanism for asserting a drop state for unexpected 
multicast data packets arriving at any enabled interface. Also, there may be included a 



wo «3/(18l451 



-9- 



PCT/US«3/(W874 



mechanism for creating forwarding state for unexpected multicast packets arriving at 
any of the physical interfaces of an enabled line processor. 

In another preferred embodiment the method may operate in protocol- 
independent multicasting sparse mode (PM-SM), wherein a processor of the data 
5 router sends a request to a neighboring router to join a group, retrieving group and 
source information, and the server module then prepares and sends to an appropriate 
client module that portion of the forwarding information base associated with packets 
expected to be received at the associated communication interfaces as a result of 
joining the group. 

10 In yet another preferred embodiment the method operates in a protocol- 

independent multicasting dense mode (PIM-DM), wherein, each time an unexpected 
multicast packet is received at a communication interface associated with a client 
module, the client module communicates information about the packet to the server 
module, wliich identifies a portion of the forwarding information base pertinent to the 

1 5 packet, and then forwards that portion of the forwarding information base to the client 
module. 

In embodiments of the invention described in enabling detail below, for the 
first time a software application is made available that sends to each client module 
only that portion of the forwarding infonnation base specific to the communication 
20 interfaces associated with the cUent module. 

Brief Description of the Drawing Figures 

Fig. 1 is a prior art diagram illustrating fabric node interconnections and 
25 upstream propagation of flow control messages. 

Fig. 2 is a diagram of a fabric card in an embodiment of the present invention. 
Fig. 3 is a diagram of a fabric network of fabric cards in an embodiment of the 
present invention. 

Fig. 4 is a diagram of a fabric card of a data router having multicasting 
30 capabiUty via a multicast port according to an embodiment of the present invention. 
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Fig. 5 is a block diagram illustrating components of the multicast port of Fig. 

4. 

Fig. 6 is a flow diagrant illustrating a packet replication process of a multicast 
fabric card according to an embodiment of the present invention. 
5 Fig. 7 is a topology diagram of a typical multi-cast data route from source to 

group. 

Fig. 8 is a block diagram illustrating components of one of the routers of Fig. 7 
practicing multicast forwarding according to an embodiment of the present invention. 

Fig. 9 is a block diagram of CC and LC communication and additional routing 
1 0 table components according to an embodiment of the present invention. 

Fig. 10 is a block diagram of the components of Fig. 9 wherein a dense mode 
is practiced. 

Fig. 11 is a process flow diagram illustrating steps for practicing PIM SM on a 
distributed processor router according to an embodiment of present invention. 
1 5 Fig. 12 is a process flow diagram illustrating steps of practicing PM DM on 

an extmded processor router accordmg to an embodiment of present invention. 

Description of the Preferred Embodiments 

20 Fig. 2 is a plan view of a fabric card 20 1 in an embodiment of the present 

invention, hi this embodiment there are nine (9) ports on each card, rather than four 
as indicated in the prior art diagram of Fig. 1. This is not meant to imply that the prior 
art is limited to four ports per node, as Fig. 1 was exemplary only. 

In the fabric card of this embodiment, as shown in Fig. 2, there are nine queue 

25 managers 209, one for each external port 205 , with each queue manager isolated from 
its connected external port by an optical interface 207. The inter-node communication 
m this embodiment is by optical links. Queue managers 209 interface with crossbar 
203, which connects each of the nine ports with the other eiglit ports internally in this 
embodiment, although these internal connections are not shown in tlie interest of 

30 simplicity. 
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Fig. 3 is a diagram illustrating a fabric having interconnected fabric cards 
according to the embodiment described above witli reference to Fig. 2. In this 
diagram one card 319 is shown connected to nine neighbor cards 301, 303, 305, 307, 
309, 3 1 1, 3 13, 3 15, and 3 17. Each of the neighbor cards is illustrated as having eight 

5 additional ports for interconnecting to fiirther neighbors in addition to the one port 
comiecting the near neighbor with card 319. It will be clear to the skilled artisan from 
this diagram that interconnection complexity escalates at a very great rate as ports and 
cards (nodes) proliferate. 

Referring now back to Fig. 2, each port on each card passes through a queue 

10 management gate 209 as indicated in Fig. 2. Each queue manager comprises a 

temporar>' storage queue with controls for managing flow in the incoming direction. 
Data traffic coming in on any one port, for example, passes through a first-in-fu-st-out 
(FIFO) queue, and the queue manager is simply enabled to discard al traffic when the 
queue overflows. There are, in this scheme, no Flow Control messages generated and 

15 propagated upstream as in the prior art. The size of each queue is set to provide 
adequate flow under ordinary, and to some extent extraordinary load conditions 
without data loss, but under extreme conditions data is simply discarded until the 
situation corrects, which the inventors have found to be less conducive of data loss 
than the problems associated with conventional flow control, which uses the upstream 

20 propagated Flow Control messages. 

In an alternative embodhnent of the present invention each queue manager on 
a card has an ability to begin to drop packets at a pre-determined rate at some 
threshold in queue capacity short of a full queue. In certain embodiments fiirther the 
queue manager may accelerate the rate of packet droppmg as a queue contmues to fill 

25 above the first threshold. In these embodiments the incidence of dropping packets is 
minunized, and spread over more traffic than would be the case if dropping of packets 
were to begin only at a fiiU queue, wherein all packets would be dropped until the 
queue were to begin to empty. 
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A distinct advantage of the queue management scheme of the present 
invention is that the intelligence required is considerably lessened, and there is no 
artificial addition to the traffic load by generating Flow Control messages. 
It will be apparent to the person with ordinary skill in the art that the 
5 embodiments of the invention described in this specification are exemplary, and may 
vary in a number of ways without departing fonn the spirit and scope of the present 
invention. For example, there may be more or fewer than nine ports and queue 
managers per card, and the size of each queue may vary. 



10 Multicasting Data Within Router Fabric 

According to another aspect of the present mvention a router fabric card 
analogous to the card of Fig. 2. Above is enhanced by virtue of added circuitry for the 
purpose of performing on-board, and in some instances, off-board multicasting of data 
15 packets. 

Fig. 4 is a plan view of a fabnc card 321 of a data router having multicasting 
capability according to an embodiment of the present invention. In this embodiment a 
MuUicast Fabric Card 321 is configured with an M-Port (multicasting port) 325. Card 
321 also has a plurality of Virtual Output Queues, one of which is illustrated in this 
20 example as a (VOQ) 329. Typically a VOQ is unplemented at each port of the card, 
although all are not shown in Fig. 4 for the sake of sunphcity. Card 321 also has a 
Crossbar Switching Facility 327 implemented thereon and nine (9) ingress/egress 
ports 323 such as were described with reference to Fig. 2 above. 

M-Port 325 is an added ingress/egress port in this embodiment, which is 
25 enhanced in this example with multicasting capability. Each port 323 on card 32 1 is, 
in a preferred embodiment, an ASIC chip. However, in certain embodiments, a chip 
set or other implementation may be used. Crossbar Switching Facility 327 is adapted 
for provision of switching fimction and negotiation between mgress and egress port 
323 of card 321. The integrated components and detail of the ftmctionality of the 
30 Crossbar Switching Facility 327 is not illustrated in this example as such detail is not 
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considered significant to the scope of this invention. The makeup of tlie routing fabric 
of a given router in this example may be assumed to contain a plurahty of cards 321. 

Virtual Output Queue (VOQ) 329 is logically illustrated between one of 
ingress/egress ports 323 and Crossbar Switching Facility 327. VOQ 329 contains a 
5 queue for ever>' egress (output) on card 32 1 including one for M-port 325 and one for 
the multicasting component of M-port 325, which is fiirther detailed below. The 
direction of data flow from VOQ 329 into facility 327 is indicated by a directional 
arrow illustrated there between. In actual practice, there is a VOQ 329 implemented 
for each of the nine ports 323 and one for M-Port 325, operating on ingress traffic at 
10 each port. Each VOQ is partitioned into a plurality of queues representing all egress 
destinations of card 321 as previously described. 

The intrinsic design of card 321 leaves provision for installing more than one 
multicast port (M-Port 325) on each card, however in this exemplary diagram, only 
one M-Port is shown, and this is deemed sufficient for tlie purpose of explaining the 
1 5 present invention, hi addition, one or more multicast ports (325) on any one card 
(321) can be activated or deactivated according to projected need. Therefore on a 
fabric card (321) witli multiple multicast ports (325), one, two, or more multicast 
ports may be activated for service dependmg on projected load and needs of the 
multicast system. When projected volume of a particular multicast assignment 
20 demands, some or all multicast ports on enhanced fabric cards within a router may be 
activated for tlie duration of the mcreased load. It is noted herein that all fabric cards 
in a router need not be enhanced for multicasting but it may be assumed that a 
plurality of cards in a fabric of any router or routers (distributed) may be multicast 
enhanced. 

25 In a preferred embodunent, a multicast assignment is orchestrated to fen out 

through fabric within a router, and such an assignment may also be disfributed to 
communicating multicast-enhanced routers dishibuted strategically throughout the 
topology of a data network. In this way, a natural load balance may be achieved and 
processing is efficiently distributed, rather than the inefficient present system of 

30 generating multiple copies at one place, and then sending all through the network. For 
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a very large assignment a plurality of multicast-enhanced routers may perform 
assigned portions of the entire project. 

Returning again to Fig. 4, data packets destined for multicasting in M-port 
325 enter card 321 as indicated by a directional arrow labeled Packets In (ingress to 
5 one of ports 323). Packets may arrive at any one of ports 323 that are coupled by port 
paths to output ports of the fabric card. Packet In represents multicast data packets 
destined for M-port 325 and are queued for M-port 325 in VOQ 329. It is again noted 
that VOQ 329 functions as a set of queues with a queue manager, and the queues are 
associated with output ports. VOQ 329 manages incoming data traffic and functions 
10 as a temporary storage queue with control for managing data flow in the incoming 

direction, hicoming data traffic is passed from an ingress port of card 321 to an egress 
port of the node as long as the queue in the path between ports is less than full as 
described with reference to Fig. 2 above. 

A data packet for multicasting is queued by VOQ 329 m the same way that 
1 5 other packets are queued, except that the queue destination is M-port 325 instead of 
the destmation of an egress (all packets identified as multicast packets are sent to the 
M-Port). In this example, data packets for multicasting pass from VOQ 329 through 
the Crossbar Switching Facility and into M-Port 325 where the data packets are 
replicated according to predetermined instmctions. hi this exemplary illustration 
20 replicated data packets are represented by the capital letters A, B , C, and D. 

Replicated data packet A-D are identical to one another except for destination address, 
which is assigned by M-Port 325 as a part of tlie replication process, according to 
mformation stored in a multicast group table (not shown) accessible to the multicast 
port. More detail about internal components of M-port 325 is provided later in this 
25 specification, particularly with reference to Fig. 5. 

M-Port 325 receives the targeted data packets, as is illustrated by a directional 
arrow emanating from facility 327 and progressing toward port 325, and replicates the 
data packet into packets A-D according to instructions. The replication of incoming 
packets mto packets with four new destinations is exemplary only, and there may be 
30 fewer or many more replications than indicated in this example. 



wo 03/081451 



-15- 



PCT/llSilJ/08874 



Port 325 assigns appropnate destination addresses for packets A-D and then queues 
the data packets for egress to the next point in their destinations as though the 
repHcated data packets were non-multicast incoming data packets. Packets A-D are 
illustrated herein as leaving port 325 back into facility 327. 
5 In this example replicated packets A, B, C. and D are routed off card 321 at 

separate egress ports as indicated by directional arrows emanating from various ports 
3-3 the arrows identified by indication of the appropriate data packets A-D and by 
element numbers 33 1, 333, 335, and 337 respectively. In this example, egress paths 
331-337 canying data packets A-D lead to ingress paths of other fabnc cards. 
10 determined by theirnew destinations. Otherfabnc cards may in mm provide further 
packet replication. If card 321 is a last card before router output, then the replicated 
packets are routed to a next router for further processing, which may, in some 
projects, include more packet replication. It is noted herem that it is not required that 
packets A, B, C, and D be routed off card 32 1 using separate paths as illustrated m 
1 5 order to practice the invention. 

m one embodiment, all packets could be routed off card 321 using a single or several 
ports The use and selection of outgoing ports depends entuely on destination 
assigmnents of the packets concerned. For example, it is not requnred that a particular 
multicast packet, which may beareplicate.be routed to multiplemulticast-capable 

.0 ports in succession for fiirther replication. In fact, a designation of unicast may be 

applied foratimetoaparticular packet causingitto be routed asanormal data packet 

until, perhaps after routing through several cards within a router, it enters a card 
wherein ftirther multicasting will be performed. At entrance to the desired card, the 
unicast designation will be stripped from the packet header of a particular packet 
25 revealingthemulticastdestinationtoanM-portonthecard. Addressing mampulation 

capability can be performed at any input port on any router card by port manipulation 

of packet headers. 

It will be apparent to one with skill in the art that card 321 may have more or 
fewer ports 323 than are illustrated in this example without departing from the spirit 
30 andscopeofthepresentinvention. Likewise, there may be more than just one M-port 



PCT/US0i/((8874 



325 integrated onto card 321. The number of both conventional ports and ports 
enlianced for multicasting, as well as their activity states during operation, is a matter 
of design and implementation. 

Fig. 5 is a block diagram illustrating various components and connectivity of 
M-Port 325 of Fig. 4 in an embodiment of the present invention. In addition to the 
multicasting role of M-Port 325 as described above, data packets not designated for 
multicasting may have ingress/egress through tliis port with requirements for 
exclusion of data in or out during periods when the multicast port is actively involved 
with multicasting/replicating duties. In this example, the fact that nonnal traffic 
camiot use port 325 during active multicasting is due to the fact that, in this 
embodiment, multicast packets are looped back into the system (card) as incoming 
packets. However, in a more enhanced embodiment, additional components may be 
added to enable both normal traffic and multicast traffic to utilize port 325 
simultaneously. 

Port 325 is illustrated with an egress path (from C-Bar to egress) and an 
ingress path (from ingress to C-Bar). These paths comprise the basic routing paths of 
port 325 for normal (non-multicast traffic). A multicast (M-Cast) engine 339 is 
provided as the replicating component of port 325. Engine 339 may be implemented 
with basic logic circuitry as an integrated part of tlie ASIC enabling port 325, or as a 
separate chip in some embodiments. It is noted herein that engine 339 is ported to 
enable receipt of data as well as communication with other port-engines on a same 
fabric card and on other multicast-capable fabric cards. 

Basic functionality in the present embodiment of the invention involves 
incoming multicast packets destined for port 325 (identified herein as Incoming 
Packets) entering port 325 from tlie Crossbar Switching Facility (327, Fig. 4) and 
delivered to engine 339 by an input line 34 1 for packet replication. 

Packets identified as packets A, B, C, and D illustrated within engine 339 are 
subject to destination address assignment by engine 339 from routing information 
stored in a table 349 also illustrated within engine 339. Table 349 contains a list of IP 
destinations of a multicast group for a given multicast project. Table 349 is 
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periodically updated and propagated between active multicast ports as was described 
with reference to Fig. 4 above. 

Multicast engine 339 replicates data packets based on instruction, in this 
example packets A-D. It is noted herein that an incoming data packet functioning as a 
5 source for replication may be dropped after replication, or may be retained with the 
same or a new destination address assigned thereto. More specifically, one of packets 
A-D may be the source packet (3 replications), or all packets A-D may be replications 
with the source packet dropped (four replications). States of addresses (taken or not) 
in table 349 are updated as used in order to provide current information to all ports 
10 during an active multicast project Table 349 is periodically updated at all multicast 
ports within a router fabric, and in some cases among multiple routers, in order for all 
ports to remain current with regard to how many replications of data packets need to 
be generated and what ultimate destinations need to be assigned to the replicated 
packets. 

1 5 Once engine 339 completes the replication and address assignment for a given 

(assigned) portion of a multicast project, replicated data packets, represented in this 
example as A, B, C, and D, are transmitted via exemplary line 343 to the ingress path 
of port 325 as incoming data packets. Packets A-D are then queued in appropriate 
sections of a VOQ (not shown) associated with port 325. Packets A-D ultimately 

20 enter Crossbar Switching Facility 327 (Fig. 4) for distribution over various paths 

according to the assigned addresses for tTie replicated data packets. It is noted herein 
that the clock speed of port 325 is essentially the same as any of ports 323 (Fig. 4). 
However, in one embodiment, the speed of replication is enhanced by using an 
increased clock speed for M-Cast Engine 339 above that of other ASIC devices in the 

25 fabric card. 

In order to maintain appropriate management of data flow through port 325, a 
Back-Pressure (BP) module 351 is provided and adapted to prevent input of new data 
into port 325 during replicating (multicasting) activity of the engine. BP module 35 1 
interfaces with M-Cast engine 339 via a control line 345 to monitor the ongoing 
30 activity of the engine. When it is determined that engine 339 is fully involved with 
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replicating and address assignment of data packets during a particular multicast 
project, BP module 351 notifies Crossbar Switching Facility (327) via a control line 
347 not to input additional new data packets for processing by the engine until the 
current effort is completed. 
5 It will be apparent to one with skill in the art that engine 339 may replicate a 

higher or lower number of data packets than the number illustrated in this example 
without departing from the spirit and scope of the mvention. The number of packets 
replicated is determmed from assignment data. In a preferred embodiment all active 
engines dunng a project receive a similar portion of a multicast project. However, in 
10 more advanced embodiments, existing network load conditions including predictive 
algorithms may be used to change multicast assignments with respect to engines, 
cards, and even routers involved. There may be many such embodiments. 

Fig. 6 is a flow diagram illustrating basic steps of data packet processing by 
the Multicast Fabric Card described with reference to Fig. 4. According to an 
1 5 embodiment of the present invention step 3 53 denotes the arrival of a data packet 
designated for multicasting. The data packet, arriving through an ingress path of a 
multicast-enabled card is queued for an M-Port analogous to port 325 of Fig. 4. The 
queuing assignment is based on destination addressing of the incoming packet. A 
dotted line illustrated in this example from Step 353 to Step 36 1 denotes a continuous 
20 monitoring of data flow to the multicasting port of step 353 by a BP Module 

analogous to module 351 of Fig. 5. As described witli reference to Fig. 5, BP module 
351 communicates to Crossbar Switching Facility (327) when port 325 is busy with 
packet rephcation and address assignment activity. 

At step 355, the multicast engine within the port of step 353 replicates the 
25 packet the necessary number of times, and assigns destination addresses according to 
a multicast group table analogous to table 349 of Fig. 5 . 

In Step 357 the multicast port of step 353 queues each replicated packet according to 
destination into a VOQ analogous to queue 329 of Fig. 4. Such a VOQ exists at each 
ingress/egress port of a fabric card. Each queued packet resides in queue according to 
30 its assigned destination address for egress from a multicast card. In Step 359 each 
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packet is routed from the fabric card along a predetermined path to its assigned 
destination for fiirther processing as required for a given multitask project. In some 
cases the destination leads to ingress at a port on another multicast card. In some 
cases, a next card will not be a multicast card. In some cases egress is to a next router 
or to a final IP destination. 

It will be apparent to one with skill in the art that the process steps illustrated 
in tlus example may be further broken down into sub-steps without departing from the 
spirit and scope of the present invention. For example, a sub-step may be included 
before step 355 for updating a multicast group table. It will also be apparent to one 
skilled in the art that the embodiments of the invention described in this specification 
are exemplary and may vary in a number of ways or configurations without departing 
from the spirit and scope of the present invention. For example, a fabric card may 
contain more or fewer than nine ports and any one or all of the ports may be 
multicasting ports. Likewise, in some embodiments, the clock speed of included 
multicasting ports may be varied and selectable depending on the load of data packet 
transmission, as previously described. 

According to an alternative embodiment, a multicasting card may be 
connected to a multicasting port of a fabric card, the multicasting card provided as an 
extemal addition, hi this case data packets for multicasting egress from the fabric 
) card into tlie multicasting card, where the replication and destination assignments are 
made, then egress from the multicasting card back mto the fabric card for re-routing 
according to the newly-assigned addresses, hi some cases using an extemal port, the 
egress of the port may be coupled to a next card having a dedicated multicast ingress 
port. 

5 The present invention may be implemented in a variety of configurations of 

interconnected fabric cards, and routers enhanced to practice the invention may be 
geographically distributed within a given network topology in order to enhance 
multicastmg capabiUty throughout the topology. One skilled in die art will recognize 
that multicasting efficiency escalates proportionally at a tremendous rate as additional 
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cards and ports are added to single routers and as similarly enhanced routers are 
distributed within a multicast region. 

Distributing Forwarding Instructions 

5 

In one aspect of the present invention, multicast forwarding information is 
distributed in parts or portions to line processors that are expecting multicast data, and 
tlie distributed data is used to build a forwarding table for the multicast data expected 

to arrive at physical poits at the target. 
10 Fig. 7 is a topology diagram of a typical multi-cast data route from a source to 

a group. As described in the background section, PIM SM requires group subscription 
to multicast data. A network topolog}' 700 comprises pluraUty 702 of network 
routers. Routers 702 may be single processor routers according to prior art 
consideration. In a preferred embodiment however, it will be assumed that at least 
1 5 some routers of topology 702 are TNRs or multiple-processor routers having multiple 
interfaces as was described in the background section above. 

Upstream from routers 702 is a source (S) system 701, wluch may be a router 
or a multicast-capable server or any other type of system capable of forwarding 
multicast data. A plurality of end systems 713-716 is illustrated on the downstream 
20 side of topology 700. Systems 713-716 are considered a multicast group (G). hi tins 
example, the multicast group (713-716) is illustrated as network-connected computer 
systems. However m other embodiments there may be other types of devices mcluded 
in the multicast group. For example. Laptop computers, hitemet-capable cellular 
telephones, personal digital assistants (PDAs) or any other known hitemet-capable 
25 device whether hardwired or wirelessly connected. 

In this example, PIM SM is practiced wherein each system (713-716) of the 
multicast group is registered in the group and receives multicast data from an 
upstream host router within group 702. A Host router is defined as a router in a path 
of multicast data traffic between a source and a final destination. Routers 703, 705, 
30 707, 708, 7 10, and 7 1 1) are illustrated as host routers in this example. Each host 
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router in this example is responsible for a certain portion of a multicast assignment 
and performs the required packet replication and forwarding functions to 
dov^^stream hosts. Host routers are responsible for forwarding multicast data toward 
a group address, which is referred to in this specification as (G). Every downstream 
5 host requests the required amount of muUicast data from upstream peers. 

The paths through which multicast data travels through topology 700 are 
represented by directional arrows. For example, host routers 707, 710, and 71 1 
subscribe to PM SM and request streams from upstream host 708, which in turn 
requests a stream from upstream host 705. In turn, host router 705 requests its data 
10 from Host 704, which requests its data from source system 701 . 

Source system 701 is, in this example, the multicast source (S) and forwards a 
smgle rr^ulticast stream to downstream host 703 . Host 703 receives the multicast data 
at a known physical upstream interface, replicates the required number of data packets 
and then forwards the multicast data to a next downstream host that has requested 
15 data, in this case, host router 705, which forwards requested data to host 708. Host 
router 708 must multicast the received data to satisfy requests from routers 707, 710, 
and 711, which in turn must satisfy the end systems 713-716 comprising the multicast 
group. 

Routers 709, 717, 718, and 706 of group 702 are not part of the multicast 
90 assigmtient described herein. That is to say that they are not receiving, replicating, or 
forwarding any multicast data. However, routers 705, 703, and 706 could join die 
multicast group at anytime if requested to do so from downstream systems. 

It will be apparent to one with skill in the art that there may be many more end 
systems and hosts involved in a multicast distribution scheme than there are illustrated 
25 in this example. The inventor deems that this simple representation adequately 
illusfrates multicast forwarding between nodes in a network to a group wherein the 
data is replicated as it is forwarded downstream to the fmal destinations. 

According to a preferred embodiment, host routers 703, 705, 707, 708, 710, 
and 711 expect multicast data to arrive for processing at specific upstream interfaces. 
30 The port addresses are included in their requests for multicast data according to G 
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calculated metrics for each host. Because tlie host routers are TNR, or otherwise 
multiple-processor routers, and it is known which upstream interfaces of these 
routers will receive multicast data, it is not necessary to consult an entire table of 
forwarding information as is the case for prior-art routers engaging in multicast 
forwarding. A goal of the present mvention is to provide a mechanism for parsing and 
distributing portions of a raulticast-forx^'arding table of a router to multicast-enabled 
upstream processors before expected multicast data arrives at the interfaces. 

Fig. 8 is a block diagram illustrating components of one of the host routers of 
Fig. 7 practicing multicast forwarding according to an embodiment of the present 
invention. A TNR 801 is illustrated in this example, and could be any one of the host 
routers of topology 702 described with reference to Fig. 7. TNR 801 comprises line 
cards, control cards and fabric cards in a distributed processor architecture as was 
described with reference to the background section of this specification. Line cards 
802-807 represent line interfaces with processors, serving as interfaces between TNR 
801 and a connected external network. An internal routing fabric 81 1 represents a 
plurality of interconnected (ported) fabric cards that make up the internal routing 
network of TNR 801 . hi tlois example, only multicast-capable fabric cards (MFC) 
812-814 are illustrated in fabric 811, although there may be fabric cards that are not 
multicast-capable. MFCs 812-814 are responsible for packet replication as described 
above with reference to priority document S/N 09/854,234. 

Aplurality of control cards (CC) 808-810 is illustrated within TNR 801 and 
are adapted to provide data distribution including protocol and boot instruction to 
fabric and luie cards as well as special packet processing requirements as described 
with reference to the background section. CCs 808-8 10, as the name implies, provide 
control functions for processor interaction and data processing fiuiction within TNR 
801. 

TNR 801 for the purpose of this example in a multicast topology has an 
upstream side and a downstream side in relation to other routers in a group relative to 
multicast data. LCs 802-804 are illustrated in this example as upstream (ingress) 
) interfaces while LCs 805-807 are illustrated as downstream interfaces (egress). One 
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wUhslalUn the artofdatarouting Will understand that allofthe illustrated in^^^^^^ 
are m practice b.-duechonal, and the one directional exan.ple is an artifice for easier 
explanation. Upstream a:.d Downstream designation of interfaces of TNR 801 are 
simplified logically to better explain the invention in terms of a multicast assignment. 
5 On the side of TOR 801 labeled Upstream, LCs 802-804 are illustrated as 

actively receiving multicast data represented herem by block arrows labeled M-Data 
In pointmg to each LC 802-804. On the side of TNR labeled Downstream, LCs 805- 
807 are illustrated as actively forwarding data out to a next node or nodes. Block 
an-ows labeled M-Data Out pointing away from LCs 805-807 represent this activity. 
10 Generallyspeakmg,mTNR801,alldataincludingmulticastdatatravelsfrom 
alinecard through fabricSllto another, or backto the same imecard before leavmg 
TNR 801 Unidirectional arrows emanating from LCs 802-804, progressing mto 
fabric 811, and arrows emanating from fabnc 811 aird tl.en progressing to LCs 805- 
807 illustrate data-forwarding in TNR 801 for dns example. 
15 In this example, multicast fo^^vardlng follows a PM protocol. In other 

examples, other similar multicast protocols maybe adaptedforim 801. in most 
prior art implementations.PIM software IS provided onsmgle-processorroutersasa 
single application ramming onamainprocessor. in this example.PIM software IS 
ixnplemented as a server/client software application wherein there are multiple client 
20 modules. 

A PIM server application 815 is provided to execute on CC 810 m this 
example CC 810 is designated to handle PM control and protocol as well as routmg 
information distribution for all designated LCS havingupstream ports tl^atwillreceive 
multicast data for processing. It is noted herein that LCS 802-807 each havemultiple 
o5 physicalports. Each LC 802-804 on the upstream side of TNR 801 has a PM client 
«thereon. LC 802 has a PIM client 816, LC 803 has a PIM client 817, and LC 
804 has a PIM client 818. It is noted that m some embodiments, all LCs within TOR 
801 mayhave an executable PM client installed. All LCs, because of their multiple 
port interfaces and bi-directional port capabilities, can be considered upstream 
30 interfaces in multicast topologies. 
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CC 8 10 has an interface capability with a database entity termed a multicast 
forwarding information base (MFIB). An MFEB contains all of the paths and 
forwarding tables for all multicast activity occurring in the topology that includes 
TNR 801. PM server 815 has access to the entire information base and in some 
embodiments may obtain tlie entire base for local storage and periodic update. When 
TNR 801 requests to join in receiving multicast data for a particular G from any 
source (*,G), PIM ser\'er 815 distributes enough data to any of the particular LCs 
having upstream ports that will receive multicast data packets for that particular G. It 
is noted herein that a plurality of physical interfaces supported on one or more than 
one LC may be aggregated as a bonded interface that is seen as a single logical 
interface in layer 3 hitemet protocols. Therefore, more than one physical port may be 
uivolved in receiving multicast data from a particular S for a particular G. 

PIM clients 816-818 are identical implementations. PIM sever 815 distributes 
data to, for example, PIM client 816 on LC 802. The distributed data includes port 
configuration (if necessary), port assignment data, internal routing path information, 
and egress port information according to G metrics. The data enables LC 802 to 
receive multicast data packets for a particular G at designated physical ports and 
forward the data tlirough TNR 801 to egress without having to consult any routing 
databases as long as the multicast packets are expected at the line interface. 
I It will be appreciated that there may be many multicast assignments in 

progress at any given time that include TNR 801 as a node in multicast forwarding for 
multicast groups. Therefore, LCs 816-818 will have differing portions of MFIB 
stored locally. In one preferred embodiment, each LC builds its own local multicast 
table from data building blocks received by respective PM clients from PIM server 
5 815. When new MFB information is added or deleted for any of LCs 8 16-8 1 8, PIM 
server 815 updates the appropriate PIM clients. 

The multicast data in sparse mode is expected because of the PIM request-to- 
join protocol. However, it is possible that unexpected multicast data packets may 
arrive at an upstream interface LC. In this case, the default treatment is for a PIM 
0 client on tlie receiving LC to check with PIM server 815 at the first unexpected 
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^ulticastdatapaclcettoarriveatanyofitsports. This, of course assu sthatS.G 
u^fonnationforthe unexpected packet does not nutchanyinfonnation on the ime 
processor. ThePIMprotocoHn the sparsemode will buildadrop entry forthe 
part^cularmutocastflowofthat packet and send ittothePMclientoftherecewmg 
5 LC Th.snotification:stem.edan™.bytheinventor.Subsequentpacketsfrom 

d.e same flow amvmg at the mterface are dropped without further consultation. In 
sparse modearouter maybe conf.gurednotto accept anymulticast data from an 

upstream source that it has not requested. 

PIM softu'are server appUcation 815 is configured to cooperate with other 
10 infon.at.onappUcat.ns(notshown)runningonTm801. PM 815 is modified by a 
software mechamsm that can parse and isolate partoftheMFIBanddistnbute the 
isolated portion to the appropriatePIMcUentrunmng on the line interface that hosts 
thephysicalports thatneed the infonnation.VVhenaLC IS bootedfor the first tin.e, 

aird is expected to receive specific (S,G) multicast data packets, then all of the 
15 requiredforwardmginfonnationissentto^hecardduringorminiediatelyafterboot. 

After the LC is operational and forwarding muhicast data then only additions and 
deletenotificationsaredistnbuted. It is noted herem that the multicast forwarding 
information heldmMFIB IS created after TNRSOUoinsaspecific multicast group. 

Therefore, the hne interfaces are prepared before actual expected data amves. 
oo Fig 9 is a block diagram illustrating CC and LC communication components 

of TNR 801 of Fig. 8 and additional routing infomiation base components accordmg 
toanembodimentofthepresentinvention. Components illustrated in this example 
that were described withreferencetoFig-Sabove are notreintroduced and have the 
san.e element numbers asinFig. 8. internal componentsofTmSOlare expanded 
OS formoreclarityinFig.9.LC804isiUustratedasanupstream(U-Stream)lme 

interfaceandLC806isillustratedasadownstrearu(D-Stream)lineinterface. Fabnc 
811 is illustrated without MFCS 812-814 described with reference to Fig. 8 above 
however they may be assumed to be present. 

LC 804 has a plurality of physical ports, 8 in this example, adapted for data 
30 communication. Four of the ports are illustrated m this embodiment as outward- 
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facing ports receiving multicast data. Three of the receiving ports are receiving 
expected multicast data packets as is illustrated by a bracket enclosing 3 directional 
arrows entering the ports and labeled M-Data In (expected). A single directional 
aiTOW entering the remaining receiving port represents receipt at the port of multicast 
5 data and is labeled M-Data In (unexpected). The port receiving unexpected multicast 
data packets has a Drop label associated therewitli indicating that all unexpected 
multicast data packets are dropped at the port. Four ports on LC 804 are inward- 
facing ports illustrated as communicating to fabric 811. Two of these ports are 
engaged in control conmiunication with CC 810, which is logically illustrated in this 
10 example as having four ports, two of which are communicating with LC 804 through 
fabric 811. Two physical ports of LC 804 are illustrated as forwarding multicast data 
into Fabric 81 1 for processing and eventual egress from the fabric into LC 806. LC 
806 is also logically illustrated with eight physical ports, four of which are interfaced 
to Fabric 8 1 1 and four of which are illustrated as egress ports to an external network. 
1 5 Multicast data egressing from fabric 8 1 1 is illustrated as entering three ports 

on LC 806 by directional arrows emanating from fabric 81 1 and entering the ports. 
Three ports on tlie external interfacing (D-Stream side) of LC 806 are illusfrated 
logically as forwarding the multicast data to a next peer destination; the described 
ports are associated by an illustrated bracket labeled M-Data Out. Two of the physical 
20 ports on LC 806 are illustrated as not active in the illustration. 

CC 810 has a port that is illustrated as communicating with an MFBB 902, 
which is analogous to the MFIB described above with reference to Fig. 8, and referred 
to as a multicast forwarding information base. MFIB 902 contains all of the routing 
and path information required for receiving and forwarding all expected multicast 
25 packets through TNR 801 to egress. MFIB 902 may be an external or an internal 

database. MFB gets multicast routing inforaiation from a larger management routing 
table (MRT) 901, which also includes all of the unicast routing parameters. Each 
database is organized under a well-known tree fomiat. An MFB entry contains S and 
G information, ingress port address information, circuit address information including 
30 MFC port addresses for packet replication, and egress port information. 
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PM sen^er 815 obtains all of the required information fiom MFB over an 
illustrated bi-directional link between MFB 902 and a port on CC 810. It is noted 
herein that all illustrations of port communication capability and communication 
bet^veen components of TNR 80 1 are meant to be logically understood and may not 

5 represent actual dedicated physical paths. 

PM server 815, in a preferred embodiment, controls identification and 
distnbution of MFB routing data to all operative LCs within "mR 80 1 that may 
receive multicast data packets, hi one embodiment, every LC has a PIM client 
estabUshed thereon whether the particular LC is mvolved m multicast forwarding or 

10 not at any given time. PIM client 8 18 on LC 804 has a dynamic data table (DDT) 903 
associated therewith. Table 903 contains all of the requu-ed mformation for 
forNvardmg of multicast packets expected to amve at the 3 ports labeled M-Datain 
(expected). DDT 903 also contams the required inibrmation for dropping all 
multicast data arriving at the port on LC 804 labeled M-Data In (unexpected). DDT 

15 903 isdynamicinthatitcanbeperiodicallyupdatedthroughPIMserver/client 

communication. Some of the mformation may be pushed to PIM client 818 and some 
may be requested. 

In the case of expected multicast data, all forwarding is performed without a 
requirement for PM client/server communication by the fact that all of the MFIB 
.0 routing information for those ports is stored locally m DDT 903. However, in case a 
multicast packet amves wherein S,G mformation does not match any current data m 
DDT 903, then PM client/Server communication is required at receipt of the first 
packet. If MFB 902 has instruction for the particular packet, then DDT 903 is 
updated with the new mfomiation. However, if the packet is deemed unexpected (not 
.5 requested by TNR 801) then it and all subsequent packet of the same flow are dropped 
as is the case with the port on LC 804 labeled Drop. Circuit ID sequences between 
ingress of LC 804 and egress of LC 806, of course vary for different multicast flows 
and for multicast packets of a same flow repUcated mto separate flows. 

In this example, two ports of LC 804 are sendmg multicast data into fabnc 81 1 
30 whileportsonLC806arereceivingtheresultingdataload. Actual packet replication 
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occurs in a preferred embodiment within fabric 81 1 by multicast-enabled fabric cards 
analogous to MFCs 812-814 described with reference to Fig. 8. Bi-directional 
arrows between CC 810 and fabric 811 and between fabric 81 1 and LC 804 logically 
illustrate PIM server/client communication and/or data distribution. 
) It will be apparent to one with skill in the art that by practicing PIM SM in a 

distributed manner and distributing only information required to complete forwarding 
of expected multicast data, much control messaging and table searching normally 
required in multicast forwarding is eliminated. Even in the case of unexpected data 
control messaging and consultation with a multicast routing table is sharply reduced. 
0 Fig. 10 is a block diagram of the components of TNR 801 of Fig. 9 wherein 

PIM DM is practiced according to an embodiment of the present invention. 
Components illustrated in this example that were described with reference to Fig. 8 
and 9 above are not reintroduced and have the same element numbers. In this 
example of dense mode (PIM DM) operation, all network-interfaced ports on LC 804 
5 are illustrated as receiving unexpected multicast data. It was described with reference 
to the background section above, that PM DM operates according to a push model 
wherein the multicast data packets are pushed to all comers of the net\vork. In this 
case all first multicast packets of respective flows entering TNR 801 are unexpected. 
PM client 818 will initiate client/server PIM communication through fabric 811 upon 
20 occunence of the fu:st multicast data packet from a particular source arriving at any 
one of the active ports. 

PIM server 815 in this case initiates consultation with MFB 902, which may 
trigger communication between MFIB 902 and MRT 901 to establish a forwarding 
state for a particular (S, G) packet. All first packets having unknown (S, G) data 
25 arriving at any port of LC 804 trigger the above mentioned client/server 

communication. PIM server 815 creates forwarding state information and distributes 
that mformation through fabric 81 1 to LC 804 where it is stored in DDT 903. All 
subsequent multicast packets arriving in the same flow now have established 
forwarding mstructions stored locally on LC 804. Therefore all subsequent multicast 
30 packets are expected and can be forwarded through to egress of TNR 801 without 
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further consultation. Because initial multicast packets are unexpected in dense mode 
operation of PM, a timeout period may be established for DDT 903 to delete old 
table entries in DDT 903 that no longer apply. 

In this particular example there are 4 ports on LC 804 receiving multicast data 
5 with3portsactivelyforwardingthedataintofabricSll. On the downstream side of 
TNR 801 all ports of LC 806 are active for receiving multicast data from fabric 811 
and 4 actively forwarding the multicast data onto the next peer. It is important to note 
herein that PIM DM does not require that TNR 801 join any multicast group. Initial 
client/server PM consultation and forwarding state creation and distribution to 
10 appropriate line mterfaces enables subsequent multicast packets to be expected and 
automatically for%varded at the interfaces without fiirther consultation. 

In actual practice of the invention a distributed processor router such as TNR 
• gOlmaybeactivelyfor^vardingunicastdataandmulticastdata. TNR 801 may also 
be adapted to run in P^M SM and PM DM modes at the same time. Because of the 
15 distributed processor arrangement and large number of ports both PM modes maybe 
simultaneously operated m a distributive fashion without switching from one to 
another. 

Fig. 11 is a process flow diagram illustrating steps for practicmg PM SM on a 
distributed processor router accordmg to an embodiment of the present invention. At 

20 step 1 100 a router analogous to TNR 801 described with reference to Figs. 8-10 

initiates a request to subscribe to a particular multicast group. At this step the group 
or G information and source or S information is known. The request includes a 
network prefix address that may represent a plurality of physical ingress ports for 
receiving multicast data packets of the type associated with the request. 

25 At step 1 101 a PIM server software analogous to PM server 8 1 5 described 

with reference to Fig. 10 above, consults with a multicast routing information base 
analogous to MFB 902 of Fig. 10 to isolate a portion of the forwarding state 
information created for the particular (S.G) multicast data packets that are expected to 
arrive at the interface identified in the request to join of step 1100. At step 1102, the 
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PM sever then distributes the required data to the appropriate line interface or 
interfaces hosting the physical ports expected to receive the requested data. 

At step 1 103, a PM client software analogous to PM client 818 described 
with reference to Fig. 10 above incorporates the distributed data to produce a local 
5 forwarding entry in a distributed data table analogous to the DDT 903 described with 
reference to Fig. 10. In one embodiment, the distributed data table is created from 
primary building blocks distributed by the PM server. In another embodiment, the 
table is already constructed before distributing to the PM client. 

At step 1 104, the PM client is notified of a first data packet of the requested 
10 (S, G), and obtains the forwarding information from the local DDT. At step 1 105, the 
packet is forwarded through to egress using the distributed table information. 

It will be apparent to one skill in the art that there may be many sub-steps 
included in tliis exemplary process widiout departing for the spirit and scope the 
present invention. For example, a subroutine can be included that handles the event of 
1 5 an unexpected multicast packet amvmg at a PM SM operational interface (LC). 

Fig. 12 is a process flow diagram illustrating steps of practicing PM DM on a 
distributed processor router according to an embodiment of the present invention. At 
step 1200 unexpected multicast packets amve at ingress line interfaces of a router 
analogous to TNR 801 described with reference to Figs. 8-10. At step 1201, a PM 
20 client operating on a line interface receiving unexpected multicast data initiates a 

request to a PM server, the request including the source and group infomiation of the 
first unexpected packet to arrive. At step 1202 the PM server initiates a request to the 
multicast forwarding information base to create a forwarding state particular to the 
packet information that initiated the request. At step 1203 the multicast forwarding 
25 information base consults with the management routing table for the infontiation 
required to create the forwarding state. 

At step 1204, the PM server distributes the created state to the PM client 
operating on the target Une interface (LC) that received the packet. At step 1205 the 
PM client stores the information in a local table analogous to DDT 903 of Fig. 10. At 
30 step 1206, the packet triggering the forwarding state creation and distribution is 
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forwarded using information and all subsequent packets of the sanie (S, G) arriving at 
the same interface or forwarded using the same locally stored information without 
further consultation with the PIM server. 

The above process steps are repeated for each differing multicast packet flow 
arriving at the router. That is to say that forwarding state information is created as 
first packets arrive so subsequent packets of the same flows can be routed using the 
locally stored DDT information. 

The method and apparatus of the present invention may be practiced on all 
distributed processor routers connected to an bitemet network or any other data- 
packet-network supportmg PIM and or other similar multicast-forwarding protocols. 
The method and apparatus of the present invention also supports such conventions as 
automated-protection-switching (APS) and multi-protocol-label-switching (MPLS). 

There are many alternative embodinrents that fall within the sphit and scope of 
the invention as well, and the scope of the invention is limited only by the claims, 
which follow. 
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What is claimed is: 

1 . A software application in a multi-processor data router in wliich a forwarding 
information base for the router is maintained, comprising: 

5 a server module; and 

one or more client modules, each client module associated with one or more 
coimnunication interfaces of the data router; 

characterized in that the server module sends to each client module only that 
portion of the forwarding infontiation base specific to the communication interfaces 
10 associated with the client module. 

2. The software application of claim 1 having one server module and multiple client 
modules. 

15 3. The software application of claim 1 wherein the operating protocol followed is 
protocol independent multicast (PIM). 

4. The software application of clami 1 wherein an assigned number of physical port 
interfaces on a line processor share a single block of forwarding infonnation. 

20 

5. The software application of claim 1 wherein the portions of multicast forwarding 
information are periodically updated at their client modules by the server module. 

6. The software application of claim I wherein the router is connected to and operates 

25 on the Internet network. 

7. The software application of claim 1 wherein more than one portion of multicast 
forwarding information blocks distributed to a single client module are stored in a 
single forwarding table locally at the line processor. 
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8. The software application of claim 7 wherein individual ones of the physical 
interfaces access their assigned forwarding instructions from the local table 
according to need. 

5 9. The software application of claim 1 further including a mechanism for asserting a 
drop state for unexpected multicast data packets arriving at any physical interface of 
an enabled line processor. 

10. The software application of claim 1 fhrther including a mechanism for creating 
10 forwarding state for unexpected multicast packets arriving at any of the physical 

interfaces of an enabled line processor. 

11. The software application of claim 1 operatmg in protocol-independent 
multicasting sparse mode (PIM-SM), wherein a proce.s.sor of the data router sends a 

15 request to a neighboring router to join a group, retnevmg group and source 

information, and the server module then prepares and sends to an appropriate client 
module that portion of the forwarding information base associated with packets 
expected to be received at the associated communication interfaces as a result of 
joining the group. 

20 

12. The software application of claim 1 operating in protocol-independent 
multicasting dense mode (PIM-DM), wherein, each time an unexpected multicast 
packet is received at a communication interface associated with a client module, the 
client module communicates information about the packet to the server module, which 

25 identifies a portion of the forwarding information base pertment to the packet, and 
then forwards that portion of the forwarding information base to the client module. 



13. A method for processing multicast data packets in a multiple-processor 
router comprising steps of: 
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(a) sending a request to an upstream router to join a multicast group, tlie 
request including an ingress interface for receiving the requested multicast data 
packets; 

(b) isolating and distributing a portion of a multicast forwarding information 
5 base to a client software module associated with the ingress interface expecting to 

receive the multicast data packets; 

(c) receiving the requested multicast data packets at the ingress interface; and 

(d) using the forwarded portion of the information base to process the received 
multicast data packets. 

10 

14. The method of claim 13 wherein in step (a) the metrics include one or more of 
network layer address information, circuit ID metrics, and physical port metrics. 

15. The method of claim 13 wherein in step (b) the multicast forwarding information 
1 5 follows protocol independent multicast (PIM). 

16. The method of claim 15 wherein in step (b) distribution is from a PM server to 

one or more PM clients. 

20 17. A method for processing multicast data packets in a multiple-processor data 
router comprisuig steps of: 

(a) receiving a first multicast data packet of a data flow for forwarding at a line 
interface of the router; 

(b) initiating and sending a request to a control processor of the router, the 
25 request containing source and group information for the received data packet and 

requesting forwarding information; 

(c) receiving the request at the control processor and consulting with a main 
routing table to build a forwarding information applicable to the data packet; 

(d) distributing the appropriate forwarding information to the requesting line 
30 interface; and 
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(e) applying, at the line interface, the distributed information to forward the 
multicast data packet and subsequent data packets of the same flow. 

18. The method of claim 17 wherein in step (b) the request is sent by a PM client 
5 application to a PIM server application. 

19. The method of claim 17 wherein in step (d) die distributed forwarding 
infonnation is stored in a local table at the line processor. 

0 20. The method of claim 1 9 wherein in step (d) the local table is updated periodically 
with new forwarding infonnation. 



21. A multicast-enabled data router, comprising: 

a plurality of processors, individual ones operating as clients and associated 
with specific ones of multiple communication interfaces, and one functioning as a 
control processor having access to a forwarding information base; and 

a software application comprismg a server module executing on the control 
processor, and one or more client modules executing on individual ones of the 
processors associated with specific communication interfaces; 

characterized in that the server module sends to each client module only that 
portion of the forwarding infomiation base specific to the communication interfaces 
associated with the client modules. 



22. The data router of claim 21 wherein the operating protocol followed is protocol 
25 independent multicast (PM). 

23. The data router of claim 21 wherem the portions of multicast forwarding 
information are periodically updated at thek client modules by the server module. 

30 24. The data router of claim 21 connected to and operating on the Internet network. 
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25. The data router of claim 21 further including a mechanism for asserting a drop 
state for unexpected multicast data packets arriving at any interface. 

5 26. The data router of claim 2 1 fiarther including a mechanism for creating a 

forwarding state for unexpected multicast packets arriving at any physical interfaces 
associated with a client module. 



27. The data router of claim 21 operating in protocol-independent multicasting sparse 
10 mode (PIM-SM), wherein a processor of the data router sends a request to a 

neighboring router to join a group, retrieving group and source information, and the 
server module then prepares and sends to an appropriate client module that portion of 
the forwarding infontiation base associated with packets expected to be received at the 
associated communication interfaces as a result of joining the group. 

15 

28. The data router of claim 21 operating in protocol-independent multicasting dense 
mode (PIM-DM), wherein, each time an unexpected multicast packet is received at a 
coiTununication interface associated with a client module, the client module 
communicates information about the packet to the ser\'er module, which identifies a 

20 portion of the forwarding information base pertinent to the packet, and then forwards 
that portion of the forwarding information base to the client module. 



29. A method for processing multicast packets, comprising the steps of: 

(a) sending, by a server module to a client module, the server module 

25 executing on a control processor and the client module executmg on a processor 
associated with an individual communication interface, a portion of a forwarding 
information base pertinent the communication interface; and 

(b) using, by the client module, the portion of the forwarding information base 
sent by the server module to process multicast packets received at the communication 
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interface without requesting information from the original forwarding information 



30. The method of claim 29 having one server module and multiple client modules. 

5 

31. The method of claim 29 wherein the operating protocol followed is protocol 
independent multicast (PM). 

32. The method of claim 29 wherein the portions of multicast forwarding infonnation 
10 are periodically updated at their client modules by the server module. 

33. The method of claim 29 wherein the method is performed in a router connected to 
and operating on tlie Internet network. 

15 34. The metliod of claim 29 further including a mechanism for asserting a drop state 
for unexpected multicast data packets arriving at any enabled interface. 

35. The method of claim 29 further including a mechanism for creating forwarding 
state for unexpected muUicast packets arriving at any of the physical interfaces of an 

20 enabled line processor. 

36. The method of claim 29 operating in protocol-independent multicasting sparse 
mode (PIM-SM), wherein a processor of the data router sends a request to a 
neighboring router to join a group, retrieving group and source information, and the 

25 server module then prepares and sends to an appropriate client module that portion of 
the forwarding information base associated with packets expected to be received at the 
associated communication interfaces as a result of joining the group. 

37. The method of claim 29 operating in protocol-independent multicasting dense 
30 mode (PIM-DM), wherein, each time an unexpected multicast packet is received at a 
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communication interface associated with a client module, the client module 
coimnunicates information about the packet to the server module, which identifies a 
portion of the forwarding information base pertinent to the packet, and then forwards 

that portion of the forwarding infomiation base to the client module. 
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